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The EVALUATION CENTER, an agency of the College of 
Education, is committed to advancing the science and 
practice of educational evaluation. More specifically, the 
purpose of the Center is to increase education s capability 
to obtain and use information for planning, programming, 
implementing and evolving educational activities. To serve 
this purpose, the Center's interdisciplinary team engages 
in research, development, instruction, leadership and 
service activities. 

HISTORY ^ ^ 

The origin of the present Center traces back to 
the establishment of the Ohio State University Test Devel- 
opment Center in 1962. Due to the urgent need for a more 
comprehensive approach to evaluation than that afforded 
by standardized testing, the Test Development Center was 
expanded in 1965 into the present Evaluation Center which 
is concerned with many modes of evaluation in addition to 
standardized testing. However, test development remains 
an important part of the Evaluation Center program. 

GOALS 

The broad objectives of the currently constituted 

Center are: . , , 

to increase scientific knowledge of educational evaluation 

and planning; 

to develop evaluation strategies and designs; 
to develop evaluation methods and materials; 
to provide instruction in evaluation; 
to disseminate information related to educational evalua- 
tion; 

to assist educationists in evaluating their programs. 
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ORGANIZATION 

To serve its complex objectives, the Center has 
developed an interdisciplinary team. Currently, the staff 
of the Center consists of fifty-four members, including 
five professorial positions, plus a varying number of visiting 
faculty. The staff and visiting professors bring expertise 
from the fields of economics, education (administration, 
curriculum and supervision, elementary and secondary 
school teaching, evaluation, mathematics, planning, re- 
search methodology, and tests and measurement), psy- 
chology, sociology, systems analysis, and urban planning. 
The Center is organized into four divisions: Administration 
and Program Development; Leadership in Evaluation; Re- 
search in EvaluaLon; and Test Development. The Center 
is administered by a director and an associate director 
for each division. 

STAFF 

Daniel L. Stufflebeam, Director 

Michael S. Caldwell, Associate Director 

Administration and Program Development 

Edwin G. Novak, Associate Director 
Research in Evaluation 

Jack M. Ott, Associate Director 
Test Development 

Blaine R. Worthen, Associate Director 
Leadership in Evaluation 
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bDO"‘LO:!r:; 

EVALUATION OF PLANNED EDUCATIONAL CHANGE 
at the local EDUCATION AGENCY LEVEL 

The advent of planned change in education, with unprecedented support 
through the expenditure of ttiillions of additional dollars, has brought about 
not only a need, but a legal requirement for evaluation. In return for Federal 
support of educational programs. Congress has legislated for an accounting for 
these funds, not dol lar-and-cents bookkeeping, but an accounting for efficacy 

of the funded program. 

This measurement of efficacy, or evaluation. Is an Infant on the educa- 
tional scene. It lacks an established body of knowledge appropriate to 
education, sufficient personnel with the necessary competencies and experience, 
and the techniques and skills to satisfy the legal requirements or needs of 

the Congress and education.^ 

A traditional view of evaluation is that it ignif ies describing 

something, in terms of selected attributes, and judging the degree of accept- 
ability or suitability of that which has been described... (that is), ..any 
aspect of the educational scene, but it Is typically (a) a total school program 
(b) a curricular procedure, or (c) an individual or group of individuals." 

An explication of evaluation as required by Title I. ESEA, relates that ",,,.to 
evaluate Is to Judge the worth, rate or value of something. Each decision that 
is made, each course of action that is chosen, even each word that Is spoken 
follows an evaluation of at least one course of action. Evaluation has taken 



’stuff lebeam, 0. L. "A Depth Study of the Evaluation . 

Theory Into Practice . Columbus, Ohio. The Ohio State University College of 

Education. June 19^>6, p. 130. 

^Hagen, Elizabeth P. and Thorndike, Robert L. ■■ 

of Educational Research . New York: The Macmillan Company, I960, p. 4U2. 
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place anytlnie something is judged good or bad, better or worse, worth contin- 
uing or discontinuing, in education, evaluation provides a basis for making 
sound decisions about educational practi^. ^ pro££di^ Ot. mine) 

These guidelines imply the use of evaluation to gather and interpret 
evidence in support of ingenuity and innovation in reaching educational 
objectives, such evidence leading to a practical decision, a judgment with 
best evidence available. A further clue to the role of evaluation is that 
......evaluation is part of the teaching process and should contribute information 

during the project period as well as at the end... (so that) .. .evaluative 
information may lead to changed methods, or changed objectives, or both."'* 

Title 1 outlines a complete cycle of educational experimentation and 
change, with an initial identification o* deficiencies in local educational 
programs, followed by the development and demonstration of effective procedures 
to alleviate these deficiencies and improve local school practices. Findings 
of validated programs would then be disseminated.^ Further testimony concern- 
ing the role of evaluation is given by Hastings, who states that "if. ..(we 
are).. .to move toward the point of basing decisions. . .on educational purposes 
and outcome, we need far more evaluative data. . .than. .. in any Instance to date." 
Hastings attributes two general purposes to evaluation: a) the collection of 



^Guide to Evaluation of Title I Projects . Washington, 0. C., U. S. Office 
of Education, October 1 Sh5^ PP. 

'‘Guidelines: Special Programs for Egucat i ^-4^ 

mentary and sSSSndary EdJcatjon Act of 6fffS%"E‘dUcafT5^1955, 

i^ oT WoIect s . Washington, D. C., U. S. Office ot Mucacion, 

p. 46 . 

5r,..ide to Evaluation of Title I Projects- Washington, D. C., U. S. Office 
of Education, October 19^6, p. 5. 
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Information to be used as feedback... for further revision of materiais^and 
methods and b) to provide information as input for decision-making..." 

The many faces ^ evaluatlon e"" 

~ it is evident that the query "What is evaluation?" will bring forth differing 
definitions and characterizations. What is not quite so apparent is that ail 
of these definitions are characterizations of evaluation, or at least, components 
of evaluation. Thus, when Stake in an evaluation report speaks or distinguish- 
ing between antecedent , transaction , and out coms data^; Thomas and Kearney 
of "Identifying target populations, and special educational needs-^, Cronbach 
of "uncovering durable relationships"^; the U. S. Office of "feedback" and 
"assessing the extent and direction of change"’°; Mooney of letting teachers 
"...say what they see to do and would like to try,. .of recording and data 
gathering and comnunicating"" ; and the others of the various roles and 
characterizations of evaluation cited earlier in thl'. paper, they are all 
speaking of activities or components of evaluation that could be categorized 
in a model-taxonomy of evaluation. The CIPP Model.-To serve as a framework 



Hastings, J. Thomas. "Curriculum Evaluation: The Why of Outcomes", 

Journal of Educational Measurement, Spring 1966, pp. 



7stake, Robert E. "The Countenance of Educational Evaluatioj;". Cente^ 
Instructional Research and Curriculum Evaluation, Uni vers y » 



for 

February I 966 . Mimeograph, 

®Thomas, Alan J. and Kearney, C. 

Theory Into Practice , Columbus, Ohio: June \9oo, pp# 105-log# 

^Cronbach, R. J. “Course Improvement Through Evaluation.*' Teacher ^ 
College Record, 64, 1963, 672-683. 

^^Guldel Ines, op cit . 

"Mooney, Ross L. "initiating a Project." Theory into Pj^tlce Colun*us, 
Ohio: College of Education, The Ohio State University, June 1966, pp. 139 I*i3. 
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for a study of evaluation, the CIPP Model of evaluation developed at The Ohio 
State University Evaluation Center is not only useful, but representative of 
the logical flow of an evaluation effort. The components of the CiPP Modei 
are Context . Input . Process , and Product evaluation. By limiting the roie 
of evaluation in this paper to its use in project innovations within the iocal 

education agency, the relationship of the modei components to educational 

13 

change becomes more apparent. 

Evaluation is a way of looking at the worid, a means of organizing the 
realities— in short, an information management system which provides focus to 
the problems and successes at hand. Much of what evaluation can do for the 
school administrator or project director-manager has been done before. Few 
administrators jump into an innovation o- change activity without some assess- 
ment of context-the extant situation. Educators usually give some thought to 
the strengths and weaknesses of alternative solution strategies, and through 
school visits, telephone calls and other informal means, keep "tabs" on "how 
the project is going." Similarly, it would not be surprising to find that the 
outcomes of the innovation are measured in some manner. However, it is not 
an exaggeration to describe these activities as being informal and non-systemlzed. 
if one were to list but some of the potential sources of information at the 
disposal of the administrator, (the school, home, community, parents, pupils, 
teachers, literature, program, and research) and the .tj^es of information that 
might be useful to the administrator (attitudes, achievement, socio-economic 



'^Conceptual ized by D. L. Stufflebeam, Director, The Ohio State University 
Evaluation Center. 

'3por a discussion of evaluation at the local, state, and Federal levels, 
see (1) Stufflebeam, og £it‘ 



4 



status, and objectives) it would be apparent that the administrator would 
necessarily develop a system to handle the ensuing chaos. And so he does, by 
giving priority, for example, to 1) information gathered easily, 2) information 
acquired with least cost, and 3) information acquired with least clutter. But, 
in so doing, the administrator may lose, among others, 1) a broad base of 
information, 2) validity and reliability of the information acquired, and 
3) source credibility. He Is roost likely losing more than he is gaining. As 
an analogy, can one picture the chief executive of a school system arriving 
at a school board meeting with a shoe box full of bills and another shoe box 
filled with requests for funds by teachers, principals, aids, etc.? No! He 
has established an accounting information system, with a budget for fund 
requests and a statement of expenditures, classified by categories, and cer- 
tified as reliable and valid by a Certified Public Accountant! 

Figure 1 depicts the f!ow of information with which the school adminis- 
trator must deal, indicating some of the sources. The decision-maker, the 
central figure, would utilize the evaluation information system to provide a 
means of organizing his information to be representative of the appropriate 
sources, such as teachers, parents and pupils. He would utilize the approach 
or component (context, input, process, product) to ensure, among other criteria 
timeliness, validity, and credibility of the information. He would develop 
an information system related to his information needs, from data collection 

through reporting. 

Thus, evaluation provides for the administrator a means of looking at, 
selecting and organizing the pertinent, relevant factors of the total Informa- 
tion flow. A description of these processes and typical activities In an 
evaluation Information system appears in Figure 2, page 9. 
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Evaluat ion-* *Svstenis Anal ys i s »**** 

These components of evaluation may be further described in a look at 
evaluation in a linear sense; 



Context 


Incut Evaluation 


Process Evaluation -J Product 


Evaluation 


(planning phase) 

ft ^ 


' Evaluation 



of Project 



context evaluation, that is. Identifying local educational program 
deficiencies. Is a "snapshot" of pre-project achievement, aptitude, socio- 
economic data, attitudes, etc., similar to the balance sheet of a business 
establishment, reporting status at a point in time and space. It is akin to 
the preliminary phase of systems analysis, a limited systems study. Produc t 
evaluation, the measurement of outcomes as related to objectives, is a slmllar 
ba lance sheet , a static report. Contrasted to these two components are those 
of input (the assessment of potential solution strategies for their probability 
of effecting desired changes within the given constraints) and process 
evaluation (monitoring activities employed in implementing the planned change). 
These latter two, input and process , deal with the dynamic elements of the 
change actlvltye 

It Is Important to establish the analogy of input and proce ss evaluation 

with the discipline of systems analysis. 

"In the most general sense systems analysis and 

can be characterized as the application of scientific ^Jhods and 

tools to the prediction and comparison of the values, effectiveness, 

and costs of a set of alternative courses of action 

machine systems. •• Systems studies trace out and assess the impact 
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of a new policy on related activities in order to predict how effec- 
tive the total set of interacting elements (a system) will be in 
performing its mission. . .viewing the new along with the old as a 
part of a set of things which function together... • System studies 
identify systems - limiting elements - a communication bottleneck, 
outdated information, or an inadequate source of supply."’^ 

It is, then, not overly difficult to view evaluation as a specialized 

flow system, a system of information management for sound decision-making by 

educators. Lankton testifies to the need for evidence to be used in decision- 
15 

making, The U. S. Office of Education states that .evaluation is part 

of the teaching process and should contribute information during the project 

period as well as at the end... (so that) .. .evaluative information may lead to 

changed methods, or changed objectives, or both. For instance, data on pupil 

attainment gathered around the midpoint of the project may show that the 

16 

original objectives were unrealistic and should be scaled down." Boerrigter 

indicates that Title I evaluation should provide information for improved 

17 

administration and evidence of adequacy* The static elements, context and 
product evaluation, are essentially of a measurement nature, while Input and 
process are the prediction and monitoring of the potential and real interactions 
encompassed by the planned change. 

A more detailed examination of the CIPP Model illustrates possible 
operational phases of the components of evaluation, and some of the potential 



^\eals, Donald W. "Heuristic Models for Systems Planning", Phi Delta 
Kappan , January I 967 , pp, 199-203. 

^^Lankton, Robert S, "Closing the Gap Between Innovation and Evaluation", 
MCME Newsletter . Volume 10, Number 1, January 1967* 

16 

Guidel ines , op ci t . 

^^Boerrigter, Glenn C. "Evaluation of Title I, Elementary and Secondary 
Act of 19^5“» mimeograph. 
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techniques and/or methodology of each phase. It should be noted that these 
entries are not exhaustive of ail techniques or methods. The information 
system phases for each component make up the horizontal rows of the chart, 
and are arranged sequentially from left to right. 

Information systems for evaluation tend ko be part icuiaristic— designed 
for a specific information need. However, within a component, such as context , 
similar systems may be utilized for like information needs especially when 
source, sample requirements and criticality are congruent. Further, some 
systems may be reused, once designed, to provide continual or repeated infor- 
mation flow. 

It is convenient to use a ’'typical’* Title l-type problem and project to 
explicate the model in operation. In an examination of inner-city results 
from a battery of achievement tests, it is found that the mean scores of inner- 
city school children on a standardized math test are two and three years below 
grade level norm. Such a use of assessment procedures can be cermed context 
evaluation. Math department personnel are then consulted, who present, from 
various sources, potential solutions to the problem, such as remedial techniques, 
curriculum changes, and changes in pupil-teacher relationships. These 
alternative solutions should be viewed in reference to the barriers to success, 
viability, sufficiency, economic costs, and other maximum-minimum variates. 

Input evaluation consists of viewing these alternative solutions in reference 
to barriers to success (i.e«, acceptance of solution by teachers, students, 
etc.), viability (operation of the proposed solution(s) in a non- control led 
field setting), sufficiency (ability of solution(s) to overcome the educational 
deficiency), economic costs (relationship of costs of proposed solutions to 
expected educational gains— that is, would a four-fold increase in costs which 
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only reduce the gap between ability and achievement by twice be “worth” the 
increase?) and other maximum-minimum variates. The cycle of postulating 
alternatives and data gathering-analysis , with a subsequent valuation of the 
alternative solution strategies, provides the necessary information for input 
evaluation and consequently, decision-making about which solution strategy 
shall be employed. Further, input evaluation identifies the potential problem 
areas and critical occurrences to be monitored during implementation of the 
solution. Thus, a process evaluation design might include monitoring of teacher 
acceptance of the solution strategy, materials utilization, real practices as 
related to desired behavior, as well as an on-going evaluation of the solution 
in meeting objectives sufficiently, or perhaps, viably. Process evaluation 
is the information management system for decision-making concerning the 
expansion, contraction, modification, clarif icstJon, termination, etc. of the 
solution strategy. Employment of process evaluation is invited interference, 
that is, the decision-makers wish to have information during the course of 
the project in order to modify plans toward a sufficient solution of the 
problem. The most effective use of process evaluation and utilization of the 
information in decision-making would ensure maximizing of outcomes, such as, 
in the case cited, the greatest possible closure of the achievement-abil ity 
gap. 

The last component In the model, product evaluation, consists of the more 
traditional product measurement, relating outcomes to objectives. The variables 
tested are dependent upon objectives, such as closure in achievement-abil ity, 
improved attitude of students toward a subject matter area, improved school 
holding power, improved attendance rate— whatever variables can be reasonably 
expected to change through introduction of the change. Not to be overlooked 
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is the measurement of concomitant variates, which could be adversely affected, 
that is, increased time, interest and attention given by the student to the 
mathematics area may be deleterious to his reading achievement. 

These are but sane of the uses of the components of evaluation, and 
certainly the future will bring more ingenious and useful approaches and tools. 

These needs and demands for evaluation, and the corresponding lack of 
theory, techniques and tools testify to the infancy of the art. Turnbull 
discussed the gap between innovation and evaluation, Lankton related problems 
experienced in Title I evaluation, and Cuba, posing the question “How do the 
(evaluation sections of) Title III proposals. . .stack up?" responded “not very 
well." In his report to Miller, University of Kentucky, Guba makes recom- 
mendations concerning Title ill evaluation which may well be appropriate for 
application to Title I evaluation. These are: 

1. Spell out very carefully the objectives for the. . .program at 
the national level. 

2. Provide adequate guidelines for the local proposer on the 
matter of evaluation. 

3. Help the local proposer understand the meaning and utility of 
each of the four kinds of evaluation; Context, Input, 

Process. and Product. 

It is not the purpose of this paper to suggest that evaluation, as herein 
conceptualized, is a new invention, nor to proffer evaluation as the panacea 
to the ills of education. Rather, it will provide a way in which to view the 



^^Turnbull, Wm. W. Editorial, NCME Newsletter , Volume 9i Number 9» 
September 1966. 

^\ankton, Robert S., op cit . 

^^Guba, Egon G. Study of Title III Activities : Report on Evaluation . 

National Institute for the Study of Educational Change, October 1, 1966. 
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complexities of innovating, and some tools and techniques with which to reduce 
these complexities to a more manageable form. Industry and the military have 
used such approaches, more familiarly termed “systems analysis*’. Proponents 
of the systems approach have suggested its applicability to education. Though 
one most often thin’KS of assembly lines and chain- of- command when industrial 
and military systems are mentioned, the focus of systems analysis is to provide 
a means to handle the increasing complexity In decision-making and the choice 
among alternative solution strategies resulting from technological innovation 
and change. 

Change can be local or global; evaluation can be microscopic or macro- 
scopic, applied within the classroom, the school, a school system, and at the 
state or Federal level. T. s potential benefits of evaluation in decision- 
making to the educator must not be overlooked or denied through fear of a 
“Federal system” of education. As conceptualized, evaluation is neutral, it 
becomes good or bad only through appl icat ion— or misapplication. 

To summarize, evaluation is a tool of the decision-maker, a way of viewing 
planned educational change. The CIPP Model is a framework for evaluation of 
change* and a means of ensuring the efficacy of that change. It Is, in short, 
a systems analysis approach to planned educational change; its purpose, to 
provide information for sound decision-making; its long-range benefits, to 
provide Improved educational opportunities for youth. 



^*Meals, Donald W. . op cit . 
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